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I.  INTRODUCTION 

In  March,  197^,  the  commander  of  the  Air  University 
(AU)  of  the  Air  Force  ordered  an  investigation  in  the  uses 
of  advanced  technology  in  extension  education.  The  goal  was 
to  find  a method  of  reducing  revision  time  and  improving  the 
instruction  of  some  370  extension  courses  with  an  annual 
enrollment  of  310,000  students.  An  appointed  interdisci- 
plinary committee  of  specialists  in  education,  psychology, 
and  computer  science  decided  upon  a pilot  project  which  would 
investigate  the  use  of  computer-based  education  (CBE)  in 
Career  Development  Courses  (CDCs).  CDCs  are  extension  courses 
taken  by  airmen  while  on  the  job  to  upgrade  their  training. 
Since  CDCs  were  written  by  authors  from  the  Air  Training 
Command  (ATC),  a cooperative  AU/ATC  project  was  proposed. 

In  August,  1974,  the  commanding  general  of  ATC  was  briefed 
on  the  project  and  gave  it  his  approval. 

The  CDCs  were  prime  candidates  for  a project  which  was 
to  show  improvement  from  the  application  of  modern  technology. 
Once  written,  the  CDC  manuals  were  published  and  distributed 
to  the  field  by  the  AU.  Because  the  medium  of  paper  is  used 
and  because  the  manuals  were  distributed  throughout  the  Air 
Force,  revisions  in  texts  are  costly  and  slow,  taking  about 
six  months  to  reach  the  field.  The  use  of  an  extensive 
computer-based  network  could  reduce  the  turnaround  time  in 
revision  and  distribution  to  the  length  of  time  it  took  to 
make  the  revision.  In  the  case  of  the  simplest  sorts  of 
revisions,  a change  could  be  available  to  CDC  users  in  less 
than  a minute. 
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rhp  followinp  three  CHF  systems  were  selected  for  the 
project : 

1)  Time-Shared  Interactive  Computer-Controlled 
Information  Television  (TTCCIT),  a television 
based  system  developed  hv  Pripham  Yoiinp  Univer- 
sity and  the  ^'itre  Corporation. 

2)  Programmed  I.op,ic  for  Automatic  'teaching  Oper- 
ations (PLATO),  a large  CPE  system  servicing 
about  1000  terminals  located  throughout  the 
United  States  with  a large  computer  at  the 
University  of  Tiiinois. 

S)  Lincoln  Terminal  Svstem  (LTS),  a microfiche 

based  teaching  machine  system  developed  by  Lin- 
coln Laboratories  at  the  Massachusetts  Insti- 
tute of  Technology. 

The  goal  of  the  project  was  to  determine  whether  one  of  these 
systems  was  sufficiently  more  attractive  than  textbooks  to 
justify  exploring  the  purchase  of  an  economical  CBF  system 
for  deliverv  of  the  CPC  wherever  appropriate. 

Before  the  project  pot  underway  the  use  of  LTS  was 
dropped  because  of  high  costs.  Thus,  in  addition  to  the 
objective  of  investigating,  the  possibility  of  using  CBE  in 
CDCs,  the  project  became  a comparison  between  the  PLATO  and 
TICCIT  systems.  So  far  as  is  known,  the  project  at  All  is 
the  only  one  in  which  the  two  systems  were  compared  side-by- 
side,  teaching  to  the  same  objectives,  and  being  staffed  by 
comparably  qualified  staffs.  This  report  focuses  on  compar- 
ing the  two  systems  based  on  the  experience  gained  at  AU. 

The  information  for  this  report  comes  from  three  major 
sources : 

1.  The  final  evaluation  report  of  the  project  by 
the  director  (Hines,  1976), 

2. . An  extensive  interview  with  the  project  officer 
of  the  AU  project. 
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■j.  The  personal  experiences  of  personnel  from  the 
Computer-based  education  Research  Laboratory  who 
conducted  the  initial  PLATO  training  at  All  and 
acted  as  programming  and  instructional  design 
consultants  for  the  duration  of  the  project. 
Additional  information  has  been  obtained  in  a telephone 
interview  with  an  employee  of  Courseware,  Inc.  which  was  the 
agency  responsible  for  the  initial  training  of  the  members 
of  the  project  using  the  TICCIT  system. 


i 


A 


II.  THE  TICCIT  AND  PLATO  SYSTEMS 

A brief  description  of  a few  relevant  characteristics 
of  both  CBE  systems  will  suffice  to  give  a background  for  a 
discussion  of  the  Maxwell  project.  The  most  basic  of  these 
are  the  factors  motivating  the  design  of  each  system.  These 
factors  along  with  pedagogical  prejudices  built  into  the  sys- 
tems determined  the  type,  as  well  as  the  manner  of  develop- 
ment, of  CBE  materials  on  each  system. 

A good  comparision  of  the  two  systems  may  be  found  in 
Bunderson  and  Faust  (1976).  Readers  wishing  a more  exten- 
sive description  of  the  PLATO  system  should  refer  to  the  art- 
icle by  Sherwood  and  Smith  (1976)  and  the  bibliography  of 
Lyman  (1977).  The  MITRE  Corporation  (1974)  provides  a 
detailed  overview  of  the  TICCIT  system. 

The  TICCIT  system.  The  TICCIT  system  was  designed  with 
the  intent  of  becoming  the  first  of  a new  generation  of  cost- 
effective  CBE  systems  (MITRE,  1974,  p.  4).  With  this  intent 
the  system  was  tailored  from  the  start  to  cut  the  costs 
associated  with  CBE  systems.  One  way  in  which  this  was  done 
was  to  base  the  system  on  off-the-shelf  hardware  components. 
These  components  were  selected  for  their  low  cost  in  1971  and 
their  costs  have  diminished  since  then  (Bunderson  and 
F aust , 1976,  p . 12) . 

Also,  at  the  outset  it  was  recognized  that  CBE  course- 
ware is  very  expensive.  In  an  attempt  to  minimize  this  cost 
as  well  as  to  provide  a good  environment  for  CBE  authoring, 
the  delivery  and  authoring  components  of  the  TICCIT  system 
were  separated.  While  it  is  possible  to  author  on-line,  the 
TICCIT  system  is  optimized  towards  instructional  delivery. 

The  authoring  is  carried  out  by  filling  out  paper  forms.  At 
a later  time,  the  information  contained  on  these  forms  may  be 
entered  onto  the  TICCIT  system  by  packagers  using  similar 
formats  on-line.  In  this  way,  authors,  who  are  not  likely  to 


come  to  the  task  of  CBF  lesson  writing  with  programming 
skills,  are  spared  from  the  complexities  of  programming. 
Moreover,  even  the  packagers  are  spared  of  this  task. 

Finally,  the  TTCCTT  system  uses  a system-implemented 
learning  strategy.  This  strategy  provides  that  the  "students 
must  be  given  a chance  to  use  learning  strategies  that  they 
develop  themselves  and  must  be  free  to  accept  or  reject  any 
strategy  advice"  (Faust,  1974,  p.  95).  This  "learner  control" 
strategy  is  incorporated  into  the  forms  that  the  authors  and 
packagers  use.  Thus,  the  authors  are  further  relieved  of  the 
responsibility  of  deciding  what  learning  strategy  is  most 
appropriate  to  their  subject  matter. 

An  early  goal  of  the  designers  of  the  TICCIT  system  was 
to  develop  a way  to  author  CBE  materials  that  would  result 
in  uniformly  high  quality  lessons  in  an  efficient  manner. 

The  fact  that  the  TICCIT  lessons  would  all  use  the  learner 
control  format  would  at  least  give  the  lessons  a uniform 
quality.  (Steinberg  (1977)  gives  a summary  of  the  research 
to  date  on  the  effectiveness  of  the  learner  control  strategy.) 
The  adoption  of  a single  lesson  strategy  also  promotes  effi- 
cient lesson  production  since  the  authors  do  not  need  to 
invest  an  effort  in  designing  for  each  lesson  the  strategy 
that  is  most  appropriate  to  the  the  subject  matter  and 
intended  audience.  The  liberation  of  those  directly  asso- 
ciated with  the  lesson  production  from  the  complexities  of 
programming  also  contributes  to  efficient  lesson  production. 

For  the  task  of  courseware  production,  the  TICCIT  system 
planners  provided  a blueprint  of  production  team  that  could 
best  carry  out  the  activities  associated  with  this  task 
(Bunderson,  1974).  The  team  consisted  of  an  instructional 
psychologist,  an  instructional  design  technician,  an  evalua- 
tion technician,  one  or  more  packaging  specialists,  and  two 
to  four  subject  matter  experts.  The  responsibilities,  and 
hence  the  requisite  qualifications,  for  each  position  on  the 
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development  team  are  precisely  defined.  With  the  exception 
of  the  subject  matter  specialists  and  the  packaging  special- 
ist, the  team  is  composed  of  personnel  trained  in  various 
aspects  of  instructional  psychology.  Bunderson  (197^1)  posits 
five  week  cycles  for  the  production  of  all  lesson  components 
and  graphic  displays.  Of  this  time,  ? to  3 weeks  are  for  the 
preparation  of  the  main  content,  and  1 to  2 weeks  for  the 
packaging  with  the  final  two  weeks  being  used  for  review  and 
revision . 

The  PLATO  system.  Although  the  PLATO  IV  system  was  also 
designed  with  cost  considerations  in  mind,  it  bears  very 
little  resemblance  the  TICCIT  system.  Instead  of  basing  the 
PLATO  system  on  off-the-shelf  hardware  components,  the  PLATO 
system's  design  was  based  on  the  premise  that  "the  technology 
of  the  1960s  was  not  capable  of  making  a significant  and 
economically  practical  contribution  to  the  nation's  educa- 
tional program"  (Alpert  and  Bitzer,  1970,  p.  1583).  Instead 
it  was  proposed  to  make  use  of  recent  developments  in  large, 
high-speed  computing  machines  and  novel,  high  performance 
graphics  terminals  which  would  be  relatively  inexpensive  when 
mass-produced  to  construct  a large-scale  system  of  as  many  as 
4000  terminals  ( Ibid . , p.  1587).  It  was  hoped  that  by  cre- 
ating a facility  capable  of  delivering  CBE  materials  to  a 
large  audience  the  high  cost  of  lesson  development  could  be 
shared  by  enough  users  to  make  such  development  cost-effective. 

A goal  of  the  system  designers  was  to  provide  the  edu- 
cational community  with  a CBE  software  system  "for  organizing 
various  teaching,  testing,  or  research  strategies"  (Ibid.  , 
p.  1587).  Thus,  the  TUTOR  language  was  developed  for  the 
PLATO  system  which  enabled  CBE  lesson  writers  to  design  their 
instruction  to  suit  virtually  any  instructional  approach. 
Because  TUTOR  provides  so  many  capabilities  for  CBE  lesson 
writers,  it  is  a complex  language.  As  a result,  learning  to 
program  TUTOR  and  actually  programming  a satisfactory  PLATO 
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lesson  has  become  in  some  environments  the  most  labor  inten- 
sive aspect  of  PLATO  lesson  development.  This  has  been 


f - observed  particularly  in  courseware  development  projects 

I ! which  did  not  have  access  to  previously  trained  programming 

talent  (Himwich,  1977). 

in  summary,  the  emphasis  in  designing  the  PLATO  system 
was  to  provide  a highly  versatile  facility  capable  of  sup- 
porting a variety  of  instructional  strategies  and  courseware 
production  efforts.  For  this  reason,  PLATO  users  may  choose 
the  manner  in  which  they  produce  their  CBE  lessons.  In  fact, 
a wide  variety  of  CBE  lesson  production  methods  have  been 
used  with  success  on  the  PLATO  system.  The  spectrum  of  these 
methods  of  production  has  the  single  author  doing  the 
instructional  design,  programming  and  formative  evaluation  to 
teams  of  several  people  dividing  the  tasks  of  lesson  devel- 
opment among  themselves  according  to  some  plan. 
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III.  PROJECT  DESIGN 

Structure 

The  major  goal  of  the  AU/PTC  CBE  project  was  to  evaluate 
the  use  of  CBE  for  the  CDCs.  A secondary  objective,  however, 
was  to  determine  which  of  the  three  media  was  the  most 
effective  in  delivering  the  CDCs.  Consequently,  it  was 
decided  to  implement  the  same  two  CDCs  in  each  medium.  The 
Food  Service  Specialist  Course  (62250)  and  the  Materiel 
Facilities  Specialist  Course  (64730)  were  selected  for  this 
purpose.  These  courses  were  to  be  redesigned  and  implemented 
separately  in  three  different  media  — programmed  texts, 

TICCIT  lessons,  and  PLATO  lessons. 

The  lessons  for  each  medium  were  to  be  written  by  three 
separate  teams  sharing  a group  of  subject  matter  specialists. 
Each  team  included  its  own  authors  and  programmers  who  were 
responsible  for  casting  the  subject  matter  into  forms  suited 
to  the  delivery  media.  To  insure  a valid  comparison  among 
the  three  media,  the  qualifications  of  the  three  staffs  were 
approximately  equal.  For  the  same  reason,  the  levels  of 
funding  for  each  medium  were  equivalent  wherever  possible. 

In  addition  to  the  three  different  media  being  consid- 
ered and  the  two  courses,  the  variable  of  student  aptitudes 
was  to  be  observed.  For  the  purposes  of  the  project,  stud- 
ents were  to  be  divided  into  thirds  based  on  the  Air  Force 
Qualifications  Tests  (AFQT). 

Objectives 

The  specific  objectives  of  the  project  are  given  in 
Hines  (1976)  as  follows: 

To  determine  the  relative  performance  on  VREs 
[Volume  Review  Examinations],  CEs  [Course  Examin- 
ations], and  SKTs  [Specialty  Knowledge  Tests]  of 
students  who  have  taken  the  conventional  hard  copy. 
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TICCIT  and  PI.ATO  versions  of  the  two  selected 
courses . 

To  determine  whether  the  updating  of  the  CDCs  (in 
two  selected  courses)  can  be  accomplished  more 
effectively  and  efficiently  with  a CAI  approach 
than  with  conventional  approaches. 

To  determine  whether  the  material  in  two  selected 
courses  can  be  effectively  and  efficiently  pre- 
sented without  the  use  of  hard  copy  text. 

To  determine  whether  student  attitudes  toward  CAI 
. ill  remain  positive  during  the  courses  of 
instruction . 

To  determine  whether  selected  individual  differ- 
ences are  more  or  less  adequately  accommodated  when 
a CAI  approach  is  used  as  opposed  to  a conventional 
approach . 

To  determine  whether  curriculum  development, 
programming,  etc.,  can  be  effectively  accomplished 
by  ATC/ECI  [Extension  Course  Institute]  personnel. 

To  determine  whether  CAI  courseware  will  demand 
more  or  less  manpower  and  time  to  prepare  and 
maintain  and  administer  than  will  conventional 
extension  courseware  (Hines,  1976,  pp . 7-8). 
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IV.  HISTORICAL  SUMMARY 

’ The  authors  and  programmers  arrived  at  Maxwell  AFB  in 

t. 

late  April,  1975-  A delay  of  more  than  two  months  in  the 
arrival  of  the  subject  matter  experts  (SMEs)  was  caused  by  a 
less-than-enthusiastic  effort  by  ATC  in  assigning  personnel. 
In  addition  to  personnel  who  were  at  the  project  site  on 

? temporary  duty  assignments  to  familiarize  themselves  with  the 

' i 

CBE  systems,  the  authoring  and  programming  personnel  included 
: j six  authors,  four  programmers,  and  one  editor.  Thirteen  SMEs 

i 

' , joined  the  project  at  various  times  after  the  arrival  of  the 

other  project  personnel. 

The  project  got  underway  on  April  28,  1975,  with  a work- 
shop on  the  writing  of  objectives.  During  this  two-week 
workshop,  a first  version  of  the  objectives  for  one  of  the 
project’s  courses  was  produced. 

The  next  six  weeks  of  the  project  were  devoted  to  fam- 
iliarizing the  TICCIT  and  PLATO  teams  with  their  respective 
media.  This  training  is  discussed  in  more  detail  in  the  sec- 
tion entitled  "CBE  Training".  By  June  20,  1975,  both  teams 
had  received  enough  formal  training  to  begin  authoring  CBE 
lessons. 

While  actual  authoring  did  begin  after  the  formal  train- 
ing, progress  in  lesson  production  was  slowed  by  the  late 
f arrival  of  the  SMEs  assigned  to  the  project.  The  first  of 

the  SMEs  did  not  begin  to  arrive  until  July,  1975,  more  than 
two  months  after  the  project  began.  It  was  not  until  seven 
months  after  the  project's  start  that  all  the  ATC  subject 
matter  experts  were  available  to  the  project. 

The  remainder  of  the  project,  until  February,  1976,  was 
spent  in  implementing  the  two  chosen  courses  on  the  three 
separate  media.  In  January,  1976,  when  about  80%  of  the  CBE 
courseware  development  had  been  completed,  the  project's 
funding  was  cut.  Consequently,  the  intended  field  test  and 


IL 


„ comparison  of  the  PLATO  and  TICCIT  systems'  instructional 

1 performance  and  cost  effectiveness  was  not  made.  However, 

^ the  fact  that  the  project's  courseware  development  phase  was 

’ almost  complete  does  allow  for  a comparison  of  the  costs  of 

courseware  development  for  the  two  CBE  systems. 

( 

t 

I 
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V.  PROJFCT  STAFFING 

* VJith  the  exception  of  the  project  administrators  and 

certain  technicians,  each  project  member  was  placed  in  one  of 
three  groups  depending  upon  the  medium  he  or  she  was  prepar- 
ing instructional  material  for.  In  order  to  provide  a basis 
for  comparison  of  the  three  media,  each  group  was  given 
! approximately  the  same  number  of  members.  For  the  same  rea- 

I 

son,  the  aggregate  qualifications  of  each  group  were  kept  as 
i as  similar  as  possible.  Deviations  from  these  intentions 

I were  caused  by  different  requirements  in  the  preparation  of 

courseware  and  limitations  in  the  entire  project's  staffing. 

TICCIT  team.  The  TICCIT  team  was  composed  of  nine  full- 
time members.  Two  members  of  the  team  were  educational 
experts  from  ECI.  They  were  college  graduates  selected  for 
the  project  partially  because  they  already  possessed  exper- 
ience in  the  preparation  of  military  training  materials.  It 
was  the  responsibility  of  the  authors  to  cast  the  subject 
matter  into  a textual  form  compatible  with  the  TICCIT  lesson 
structure.  In  addition  to  their  authoring  function,  they 
also  reviewed  the  lessons  with  the  SMEs  for  accuracy  and 
educational  soundness.  Each  one  of  these  author-reviewers 
was  responsible  for  one  of  the  two  courses.  One  of  the 
TICCIT  author-reviewers  had  three  SMEs  working  with  him;  the 
other,  2 SMEs.  The  SMEs  were  enlisted  personnel  who  had 
extensive  experience  in  the  field  and,  in  some  cases, 
instructional  experience  in  their  specialty.  In  addition  to 
providing  subject  matter  expertise,  the  SMEs  offered  sugges- 
tions on  the  best  methods  of  fitting  the  subject  matter  to 
the  TICCIT  instructional  strategy. 

The  final  members  of  the  TICCIT  team  were  the  two  mem- 
bers who  converted  the  text  produced  by  authors  and  SMEs  into 
TICCIT  lessons.  These  were  the  only  members  who  worked  dir- 
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ectly  with  the  TICCIT  hardware  system.  These  entry  special- 
ists were  young  officers  with  little  experiehce  with  the 
subject  matter  or  instructional  design. 

PLATO  team.  The  structure  of  the  PLATO  team  was  very 
similar  to  that  of  the  TICCIT  team.  The  PLATO  team  had  two 
members  from  ECI  who  were  primarily  text  authors  and  lesson 
reviewers;  each  of  these  was  responsible  for  one  of  the  two 
courses  being  implemented  on  the  PLATO  system.  Each  author 
had  two  SMEs  to  help  him  or  her  organize  the  CBE  lessons  and 
maintain  technical  accuracy.  The  lessons  in  preliminary  form 
produced  by  each  of  these  authoring  were  translated  into  a 
PLATO  lesson  by  two  PLATO  programmers.  Unlike  the  authors 
and  the  SMEs,  the  programmers  prepared  PLATO  lessons  for 
either  one  of  the  two  courses. 

Although  the  structure  of  the  PLATO  team  was  the  same  as 
that  of  the  TICCIT  team,  roles  became  somewhat  more  blurred 
on  the  PLATO  team.  During  the  initial  PLATO  training  the  ECI 
authors  for  the  PLATO  team  were  given  the  same  programming 
training  as  the  team's  programmers.  This  training  was 
thought  to  be  important  so  that  the  authors  would  be  aware  of 
the  varied  instructional  capabilities  of  the  PLATO  system. 
Thus,  as  designers  of  the  PLATO  lessons,  they  would  know  that 
they  had  considerable  flexibility  in  selecting  an  instruc- 
tional approach  was  appropriate  to  their  subject  matter.  The 
PLATO  programming  training  did  make  the  authors  aware  of  many 
of  the  PLATO  system's  instructional  capabilities  while  si- 
multaneously making  them  modestly  competent  PLATO  programmers. 
Thus,  though  the  more  sophisticated  programming  was  done  by 
the  team's  designated  programmers,  the  authors  themselves 
were  able  to  program  some  of  the  easier  parts  of  their  les- 
sons by  themselves. 

The  case  was  similar  for  the  SMEs  of  the  PLATO  team. 
Although  they  arrived  after  the  formal  PLATO  training  had 
been  completed,  they  received  an  informal  on-the-job  intro- 
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duction  to  the  PLATO  system  and  the  rudiments  of  the  TUTOR 
ianpuage  from  other  members  of  the  PLATO  team.  As  a result, 
they  were  also  able  to  perform  easy  programming  tasks.  These 

i 

‘ tasks  were  seldom  more  complex  than  the  arrangement  of  a 

PLATO  display.  Nevertheless,  through  the  programming  efforts 
^ of  the  SMEs  and  the  authors,  the  PLATO  programmers  were  able 

1 to  devote  themselves  to  the  more  challenging  programming. 

i 
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VI.  CBE  TRAINING 

TICCIT  training.  Early  in  1975,  the  two  project  person- 
nel who  were  to  handle  the  answer-processing,  debugging,  and 
production  for  the  TICCIT  team  were  given  temporary  duty 
assignments  for  seven  weeks  to  familiarize  themselves  with 
the  TICCIT  system.  This  was  carried  out  at  North  Island  NAS, 
California.  Their  training  included  TICCIT  software  charac- 
teristics and  their  implications  for  instructional  materials, 
terminal  data  entry  techniques,  and  the  development  of  some 
TICCIT  instructional  materials  as  a training  exercise. 

In  May,  1975,  two  representatives  from  Courseware,  Inc. 
began  a training  session  at  the  project  site.  This  training 
focused  on  structuring  instructional  materials  for  the  TICCIT 
system.  Before  the  end  of  the  first  week  of  this  session, 
the  trainees  were  writing  instructional  segments  that  were 
incorporated  without  change  in  TICCIT  lessons. 

After  the  first  week,  the  TICCIT  hardware  was  installed. 
To  train  the  project  personnel  to  use  it.  Courseware,  Inc. 
sent  a representative  skilled  in  this  area  as  a replacement 
for  one  of  the  original  representatives  and  a graphics 
specialist.  During  the  next  two  weeks,  the  training  consisted 
creation  and  processing,  TICCIT  familiarization,  and  data 
entry.  Simultaneously,  the  original  training  in  courseware 
development  and  answer  processing  was  continued. 

An  additional  week  of  training  was  given  to  the  TICCIT 
team  about  six  months  later.  At  that  time,  most  of  the  tra 
struction  dealt  with  the  refinement  of  instructional  devel- 
opment techniques  and  assistance  with  individual  problems. 

The  training  of  the  TICCIT  team,  then,  consisted  of 

of  familiarization  with  the  system  for  two  mem- 
TICCIT  team  and  three  weeks  of  training  by 


seven  weeks 
bers  of  the 
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Courseware,  Inc.  personnel  at  the  project  site.  The  project 
officer  was  not  well  pleased  with  Courseware's  training  and 
expressed  the  opinion  that  the  training  materials  seemed  to 
be  in  the  developmental  stages.  '^e  partially  attributed  the 
slow  development  times  of  the  TICCIT  courseware  to  poor 
training  from  Courseware  (Hines,  1976,  p.  ^<1  ) . The  employee 
of  Courseware  who  was  interviewed  agreed  that  the  training 
offered  to  the  project  personnel  was  not  well  organized. 

PLATO  training.  The  bulk  of  PLATO  training,  was  carried 
out  in  a two  week  session  in  June,  1975.  This  initial  train- 
ing and  the  follow-on  training  was  conducted  by  members  of 
the  Military  Training  Centers  (MTC)  group  at  CERL.  The 
training  and  instructional  materials  that  were  presented  to 
the  Maxwell  PLATO  team  had  already  been  extensively  refined 
by  use  in  training  over  250  PLATO  authors.  In  addition,  on- 
line PLATO  materials  written  by  the  MTC  group  to  accompany 
its  hardcopy  training  materials  had  been  used  by  over  10,000 
users.  This  experience  had  enabled  the  MTC  group  to  develop 
a well  polished  training  program  that  taught  the  essentials 
of  PLATO  authoring  and  programming  (Francis,  1976). 

During  the  first  week  of  the  Maxwell  PLATO  team's  train- 
ing, a member  of  the  MTC  group  introduced  the  team  to  the 
PLATO  system  and  to  a basic  subset  of  the  TUTOR  language,  the 
programming  language  used  on  the  PLATO  system.  The  30  TUTOR 
commands  that  were  presented  were  chosen  with  the  aim  of 
introducing  the  trainees  to  the  capabilities  of  the  TUTOR 
language.  Once  these  had  been  mastered,  the  trainees  could 
use  an  array  of  on-line  reference  materials  to  supplement 
their  basic  knowledge  as  the  need  arose.  At  the  end  of  the 
first  week  of  PLATO  training  both  members  of  the  PLATO  team 
who  had  been  designated  as  programmers  had  completed  the 
TUTOR  portion  of  the  PLATO  training.  Also,  one  of  the  two 
members  who  were  to  become  instructional  designers  or  authors 
has  completed  the  training;  the  other,  being  available  for 
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training  only  half-time,  was  somewhat  behind  the  others. 

The  second  week  of  training  emphasized  instructional 
design  techniques  appropriate  to  PLATO  instructional.  This 
training  included  PLATO  lesson  planning  and  design,  tech- 
niques of  computer-student  interaction,  and  computer-based 
testing.  During  this  week,  the  PLATO  instructor  supervised 
the  planning,  design  and  beginning  writing  of  the  first  PLATO 
lessons  of  the  trainees. 

About  two  months  after  the  initial  PLATO  training  two 
CERL  staff  members  returned  to  offer  some  additional  consul- 
tation. One  of  these  staff  members  taught  the  programmers 
some  needed  sophisticated  programming  techniques.  Since  at 
that  time  initial  versions  of  the  project's  first  PLATO  les- 
sons were  ready,  the  other  trainer  introduced  the  PLATO  team 
to  lesson  review  methods  and  other  techniques  used  in  the 
formative  development  of  CBE  lessons.  This  last  follow-on 
training  period  lasted  three  days. 

In  addition  to  the  training  given  the  Maxwell  PLATO 
team  at  the  project  site,  MTC  personnel  could  always  be  con- 
sulted via  the  PLATO  system  on  programming  and  instructional 
design  problems.  PLATO  communication  features  enabled  MTC 
staff  members  to  assist  the  Maxwell  project  quickly  with  pro- 
blems as  they  arose.  The  reader  may  obtain  a more  detailed 
description  of  both  the  type  of  training  and  sort  of  services 
offered  the  Maxwell  project  by  the  MTC  group  by  reading  the 
sections  entitled  "CBE  Author  Training"  and  "MTC  Liaison"  in 
Himwich  ( 1 977  ) . 
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VII.  CHE  COURSEWARE  DEVELOPMENT 

In  order  to  maintain  comparability  between  the  TICCIT 
and  PLATO  systems,  the  CRF  lesson  development  procedures 
employed  by  each  group  were,  within  the  constraints  imposed 
by  systems'  differences,  the  same.  Figure  1,  which  is  an 
adaptation  of  a graphic  representation  of  the  CBE  lesson 
development  procedures  from  Hines  (1976),  outlines  the  paral- 
] el  lesson  development  procedures  employed  by  the  two  CBE 
teams  as  well  as  their  similar  structures.  This  section  dis- 
cusses the  procedures  indicated  in  Figure  1 and  the  differ- 
ences in  these  procedures  which  were  due  to  intrinsic  differ- 
ences in  the  CBF  systems. 

Pasic  development  procedures.  Lesson  development  began 
with  the  authors  and  their  SMEs.  Figure  1 shows  four  author- 
SME  groups,  two  PLATO  and  two  TICCIT  groups.  For  either  the 
TICCIT  or  the  PLATO  team,  each  of  the  author-SME  groups  was 
responsible  for  writing  the  instruction  for  one  of  either  the 
Food  Specialist  or  the  Materiel  Facilities  Course.  (The 
objectives  for  both  courses  were  drawn  up  prior  to  the 
authoring  stage  so  that  each  team  was  writing  instructional 
materials  for  the  same  objectives.) 

Following  the  writing  of  a segment  of  instruction,  the 
authors  and  one  SME  reviewed  the  material  for  instructional 
soundness  and  subject  matter  accuracy.  Depending  upon  the 
outcome  of  this  review,  the  segment  was  either  passed  on  to 
the  programmers  for  implementation  on  the  CBE  systems  or  was 
returned  to  the  appropriate  author-SME  group  for  rewriting. 
Generally,  at  thi.s  stage,  rewriting  only  involved  small 
editorial  changes. 

Two  programmmers  in  each  team  were  responsible  for  con- 
verting the  text  to  CBE  lessons  Once  the  lessons  were  in  a 
usable  form  o.i  the  TICCIT  or  PLATO  system,  they  were  reviewed 
once  again  this  time  by  the  author  and  the  two  programmers. 
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Depending  on  the  results  of  the  review,  either  portions  of 
' the  lesson  were  rewritten  or  reprogrammed  or  were  declared 

<■  to  be  finished.  If  the  lesson  was  not  considered  to  be  fin- 

ished the  process  of  authoring  and/or  programming  and  review- 
ing continued  until  reviews  indicated  no  changes  were  nec- 
essary. Most  of  the  lessons  or  segments  of  lessons  were 
reviewed  only  twice. 

■j 

' The  basic  lesson  development  procedure  that  was  selected 

for  use  by  the  project  loosely  resembles  one  that  the  MITRE 
' Corporation  recommends  for  TICCIT  courseware  (MITRE,  197^, 

p.  ^41).  However,  it  was  adopted  for  the  PLATO  lessons  in 
order  to  obtain  a better  comparision  between  the  two  systems. 
Nevertheless,  despite  differences  in  the  systems  themselves 
and  the  consequent  differences  between  the  requirements  for 
personnel  using  them,  the  procedures  for  developing  lessons 
on  the  two  systems  were  formally  identical  with  some  varia- 
tion in  their  implementation  for  each  system. 

Differences  in  authoring.  As  discussed  in  the  section 
entitled  "The  TICCIT  and  PLATO  Systems",  the  TICCIT  system 
used  the  instructional  strategy  of  learner  control  as  the 
single  strategy  available  to  authors.  Thus,  TICCIT  authors 
have  only  to  mold  their  subject  matter  to  this  strategy. 

Once  this  has  been  done,  authors  are  provided  with  forms  on 
which  they  may  format  the  text  for  later  entry  on  the  TICCIT 
system.  For  this  purpose,  the  authors  were  provided  with 
paper  forms  for  later  use  by  data  entry  specialists  (MITRE, 
1974,  p.42).  On  the  other  hand,  the  PLATO  system  was 
designed  so  that  an  instructional  designer  must  choose  the 
learning  strategy  that  seems  best  suited  to  the  subject  mat- 
ter and  students.  Not  only  must  the  author  select  a general 
strategy,  he  or  she  must  decide  upon  the  specifics  of  adapt- 
ing the  strategy  to  a given  subject  matter.  Thus,  the 
authoring  of  PLATO  lessons  requires  some  expertise  in 
instructional  design  to  obtain  satisfactory  results  whereas 


21 


TICCIT  authors  need  very  little  design  experience.  Again, 
the  quality  of  a PLATO  lesson  may  vary  widely  depending, 
among  other  things,  upon  the  author's  lesson  design,  while 
TICCIT  lessons  are  more  uniform  in  format  and  quality  inde- 
pendent of  the  lesson's  author  or  subject  matter. 

Although  the  PLATO  system  presents  an  instructional 
designer  with  great  freedom,  the  Maxwell  PLATO  lessons  do  not 
display  much  diversity  in  instructional  approaches.  Most  of 
the  PLATO  lessons  are  straight-forward  tutorials.  Still, 
there  is  some  variation  in  the  text-question-text-question 
pacing  and  in  the  presention  of  end-of-lesson  criterion  test 
in  the  lessons.  These  superficial  differences  are  due  to 
differences  in  subject  matter  and  individual  styles  of  the 
authors.  Whether  conscious  or  not,  the  adoption  of  a single 
instructional  strategy  prevented  the  project  from  demonstrat- 
ing the  full  instructional  capabilities  of  the  PLATO  system. 
However,  such  a demonstration  was  not  appropriate  to  the  sub- 
ject matter  of  the  courses  and  would  have  needlessly  increased 
courseware  development  times. 

Differences  in  programming.  The  term  "programming"  is  a 
misnomer  in  the  case  of  TICCIT.  In  fact,  the  form-oriented 
authoring  approach  is  intended  to  keep  authors  separated  from 
programming  (MITRE,  197^,  p.  5).  The  forms  that  TICCIT 
authors  fill  out  are  transferred  to  similar  forms  displayed 
at  the  TICCIT  terminal.  This  can  either  be  done  by  the 
author  or  an  entry  clerk.  In  neither  case  is  any  programming 
skill  needed.  The  data  that  is  entered  in  this  way  automati- 
cally completes  an  already  written  skeletal  program  to  make 
a TICCIT  program  for  a lesson.  At  some  time  after  data 
entry,  an  instructional  segment  consisting  of  the  manually 
entered  data,  text  and  the  pre-programmed  package  may  be 
viewed  as  a lesson. 

The  programming  needed  for  a PLATO  lesson  is  frequently 
as  complex  as  the  "programming"  needed  for  a TICCIT  lesson  is 
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simple.  The  language  used  on  the  PLATO  system.  TUTOR,  is  a 
rich  language  containing  over  250  commands.  These  commands 
fall  into  five  large  groups;  display,  calculation,  branch- 
ing, answer  judging,  and  data  collecting.  Generally,  a PLATO 
programmer  is  familiar  with  only  the  most  basic  commands  in 
each  of  these  groups.  Usually,  however,  the  task  of  program- 
ming a lesson  will  require  several  commands  with  which  the 
programmer  does  not  possess  a working  knowledge  of. ^ Thus 
the  PLATO  programmer  must  also  be  familiar  with  the  capabil- 
ities of  TUTOR  in  each  group  so  he  will  have  some  idea  of 
what  options  he  has  in  programming  a lesson  and  how  easily 
these  options  may  be  realized. 

The  Maxwell  PLATO  programmers  were  able  to  avoid  the 
cost  of  using  several  new  TUTOR  commands  each  time  they  began 
a new  PLATO  lesson.  The  use  of  a similar  instructional 
strategy  in  each  lesson  not  only  reduced  the  time  needed  for 
instructional  design,  it  also  minimized  the  new  programming 
skills  needed  for  each  new  lesson. 

Comparative  development  costs.  Despite  the  fact  that 
the  tasks  of  authoring  and  programming  are  less  complex  on 
the  TICCIT  system  than  on  the  PLATO  system,  the  costs  of 
developing  comparable  segments  of  instruction  were  practi- 
cally the  same  for  each  system.  Table  1 gives  a breakdown  of 
these  costs.  These  data  reflect  the  manpower  that  was  needed 
to  produce  about  80%  of  the  CBE  lessons  for  each  of  the  two 
courses  and  includes  the  training  times  on  each  system 
(Hines,  1976,  p.39).  Also,  the  figure  for  PLATO  programming 
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1a  PLATO  lesson  may  be  written  using  only  a handful  of  com- 
mands. However,  economy  in  the  use  of  TUTOR  is  generally 
reflected  in  lower  lesson  quality. 
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Table  1 

Comparision  of  manpower  costs  for  producing  CBE 
lesson  material  on  the  PLATO  and  TICCIT  systems 
based  on  the  production  of  3?  student  contact  hours 

for  each  system. 


Activity 

CHK  .System 

PLATO  TICCIT 

Imh/sch)^  (mh/.sch)^ 

Authoring  A Reviewing 

Graphics  Production 

Programming 

141  150 

_-b 

HI  bO 

Total  manhours/student  contact 

. hour  222  24b 

aMan  Hours/Student  Contact  Hour 
^The  time  spent  in  producing  graphics  was  not 
separated  from  the  programming  costs. 

time  includes  the  time  needed  to  produce  graphics. 1 After  the 
two  courses  had  been  reorganized  for  the  purposes  of  the  pro- 
ject, the  Food  Specialist  Course  would  required  24  student 
contact  hours;  the  Materiel  Facilities  Course,  15  hours 
( Ibid . , p.40).  Thus,  about  32  student  contact  hours  of  CPE 
lessons  had  been  produced  for  each  of  the  two  systems. 


10ne  of  the  PLATO  author  aids  allows  the  programmer  to 
design  and  adjust  a display  at  the  terminal.  Once  the  dis- 
play is  considered  satisfactory,  the  PLATO  system  automati- 
cally generates  the  programming  needed  to  reproduce  the  dis- 
play. 
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Hines  (1976)  explains  the  major  variations  in  production 
times  as  follows: 

TiCCIT  required  more  time  for  authoring  and  reviewing 
due  to  personnel  qualifications  and  poor  quality 
training  by  the  civilian  contractor.  The  extra  time 
spent  in  designing  TICCIT  graphics  arose  from  the  fact 
that  the  drawings  had  to  be  done  on  paper,  entered 
into  the  system  by  means  of  a digitalization  process, 
and  then  edited  on-line  (Hines,  1976,  p.  41). 

in  other  military  PLATO  sites,  the  greatest  manpower 
costs  for  lesson  production  have  come  from  programming  and 
activities  associated  with  programming.  At  the  United  States 
Army  Ordnance  Center  and  School  (USAOC&S)  PLATO  project,  for 
example,  these  activities  took  about  71%  of  the  total  lesson 
development  time  (Himwich,  1977).  That  the  Maxwell  PLATO 
lessons  were  programmed  at  half  the  relative  cost  of  the 
USAOC&S  lessons  is  striking  and  may  be  due  to  the  high  com- 
petence of  the  Maxwell  programmers.  Of  more  interest,  how- 
ever, is  the  fact  that  the  Maxwell  PLATO  and  TICCIT  lessons 
took  about  the  same  amount  of  time  to  program.  That  the  pro- 
gramming costs  of  one  system  that  had  reduced  programming  to 
form-filling  should  be  comparable  with  another  system  which 
required  boiia  fide  programming  of  a frequently  complex  nature 
is  surprising. 

The  same  surprise  is  attached  to  the  generally  compar- 
able development  times  of  lessons  for  each  system.  With  its 
built-in  instructional  strategy,  simplified  authoring,  and 
its  design  intent  to  provide  cost-effective  delivery  of  CBE 
materials,  one  might  have  expected  that  cost  of  lesson  devel- 
opment, one  of  the  biggest  obstacles  to  this  intention,  would 
be  less  than  those  for  comparable  lessons  developed  for  the 
PLATO  system. 
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